Layered security in digital watermarking

ABSTRACT

A media object authentication system uses layers of security features based on digital watermarks embedded in media objects. The system generates a first digital watermark with a message payload carrying data about the object, such as a hash of text data printed on the object. The first digital watermark is combined with a content signature derived from features of the media object, such as frequency domain attributes, edge attributes, or other filtered version of the media signal (e.g., image photo on a secure document) on the media object. This combination forms a new digital watermark signal that is embedded in the host media object. To verify the object, the digital watermark payload is extracted and compared with the data about the object. The combined digital watermark and content signature is also evaluated to authenticate the media signal on the media object.

RELATED APPLICATION DATA

This application is a continuation of application Ser. No. 10/158,385,filed May 29, 2002 (now U.S. Pat. No. 7,519,819).

TECHNICAL FIELD

The present technology relates to digital watermarking andauthentication of media objects.

BACKGROUND AND SUMMARY

Digital watermarking is a process for modifying physical or electronicmedia to embed a hidden machine-readable code into the media. The mediamay be modified such that the embedded code is imperceptible or nearlyimperceptible to the user, yet may be detected through an automateddetection process. Most commonly, digital watermarking is applied tomedia signals such as images, audio signals, and video signals. However,it may also be applied to other types of media objects, includingdocuments (e.g., through line, word or character shifting), software,multi-dimensional graphics models, and surface textures of objects.

Digital watermarking systems typically have two primary components: anencoder that embeds the watermark in a host media signal, and a decoderthat detects and reads the embedded watermark from a signal suspected ofcontaining a watermark (a suspect signal). The encoder embeds awatermark by subtly altering the host media signal. The readingcomponent analyzes a suspect signal to detect whether a watermark ispresent. In applications where the watermark encodes information, thereader extracts this information from the detected watermark.

Several particular watermarking techniques have been developed. Thereader is presumed to be familiar with the literature in this field.Particular techniques for embedding and detecting imperceptiblewatermarks in media signals are detailed in the assignee's U.S. Pat.Nos. 6,122,403 and 6,614,914, which are hereby incorporated byreference.

One application of digital watermarking is for the authentication ofphysical and electronic media objects, like images, video, audio, andprinted media. There are a variety of ways to authenticate theseobjects. One way is to embed a predetermined watermark in the object. Ifa reader detects this watermark in an object, then the detection of thewatermark is an indicator of its authenticity.

Another way to authenticate the object is to embed information about theobject or the bearer of the object (e.g., in photo ID or other securedocuments). If the reader extracts this information from the watermark,and it matches information on the object or about the bearer, then thecomparison this information is an indicator that object is authenticand/or the bearer of the object is valid.

To undermine the authentication function of the digital watermark, ahacker might try to re-create the watermark in a fake media object.

The present technology provides a method for authenticating electronicor physical media objects using digital watermarks.

One aspect of the technology is a method for creating a media object forauthentication. This method computes a hash of information on theobject, and generates a pattern from the hash. It also computes acontent signature from a media signal in the media object. It thencombines the content signature and the pattern to form a contentdependent pattern. Finally, the method embeds the content dependentpattern as a digital watermark into the media object.

One specific application of this method is to create secure documentsthat may be authenticated automatically. For example, the media objectmay comprise a photo ID or other secure document, where the hash iscomputed from data on the document and the content signature is derivedfrom features of the photo or other image on the document. The methodapplies to other physical and electronic media objects. The hash may becomputed from information in the media object, which is easilyinterpreted by a viewer or listener of the rendered object, or may becomputed from information relating to the media object.

Another aspect of the technology is a related method of authenticating amedia object using a digital watermark embedded in the media object.This authentication method providing a first pattern, either from anexternal source (e.g., user input, system memory, etc.) or derived froma digital watermark embedded in the object. The method also derives acontent dependent signature from a media signal in the media object. Itthen combines the content dependent signature and the first pattern toform a content dependent pattern. Finally, it measures the contentdependent pattern embedded as a digital watermark in the media signal toprovide a measurement of authenticity of the media signal.

Further features will become apparent with reference to the followingdetailed description and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a digital watermarking embedder used tocreate watermarked objects that are authenticated in multiple ways.

FIG. 2 is a diagram illustrating a method for authenticating mediaobjects created using the method shown in FIG. 1.

DETAILED DESCRIPTION

FIG. 1 is a diagram illustrating a digital watermarking embedder used tocreate watermarked objects that are authenticated in multiple ways. Thisdiagram shows a variety of techniques to provide layers of security in amedia object. The implementer may choose to use one or more combinationsof the elements illustrated in the diagram, such as a hash carried in awatermark, a content dependent watermark, a content signature carried ina watermark, etc. We will illustrate how these functions of the digitalwatermark may be integrated into a single watermark or in separatedigital watermarks. While the media object generally encompasses images,video, audio, and physical objects, we illustrate the method through theuse of examples of security documents that carry images embedded withdigital watermarks.

As shown in FIG. 1, the input to the embedder is an input media signal100. In our example of a security document, this input signalcorresponds to an image to be printed on the security document.

The embedder computes a signature of the media object (102) bycalculating a set of features of the media signal in the media object.Preferably, the features are selected such that they are likely to berelatively unchanged through a distortion channel that the object isexpected to pass through. In the example of a security document, thisdistortion channel includes printing, scanning (to capture a digitalimage for authentication as shown in FIG. 2), normal wear and tear,soiling, geometric distortion, etc. The features and robustness of thedigital watermarks may be adapted to survive or degrade in response todistortions that result from intentional manipulation. For example, ifan intentional manipulation occurs, such as scanning and re-printing ofa security document on a desktop scanner/printer, this manipulation mayrender the feature severely distorted and/or the digital watermarkun-readable, which serve as indicators that the document is a fake. Aplurality of digital watermarks and signal feature metrics may be usedto measure evidence of such manipulation. These metrics may be usedalong with robust digital watermarks that carry additionalauthentication information as described below.

Examples of features from which the signature are derived for an imageinclude: edge pixels detected using an edge detection filter, frequencycoefficients (e.g., low frequency coefficients of blocks in the image),relationships among neighboring pixels (e.g., differences betweenneighboring pixel values, computed using a filter that returns thedifference or sign of the difference between a pixel and the average ofits neighbors), etc. In one implementation, we use these features togenerate a binary antipodal signal of [1, −1] corresponding to locationswithin the signal to be watermarked. The antipodal signal is a vectorwhere the elements having a value of 1 represent a location of thefeature (or location where feature meets a criteria, such as above athreshold, local maxima/minima), while the−1 represents absence of thefeature (or location where the feature does not meet the criteria).

The antipodal signal can be used to embed features of the host inputsignal into the digital watermark, such as selected low frequencycoefficients. In one implementation, for example, the embeddercalculates the signature by taking a frequency transform (e.g., aDiscrete Cosine Transform) of an M by M block of the host image signal,and then quantizing the lowest frequency N by N coefficients (except DC)to 1 or−1 by performing a threshold comparison with their median value(greater than median assigned to 1, and less than median assigned to−1).This results in a binary antipodal signature of length (N×N−1), whoseelements are mapped to the M ×M locations in the original M by M block.Note that N is less than M, so the elements of the signature areredundantly mapped to the M ×M samples in the M by M block. A similarprocedure is repeated for other M by M blocks of the host media signal.

Next, the embedder computes a content dependent watermark, CW, as afunction of the signature and a basic watermark pattern, W (108) Likethe signature, this basic watermark pattern is also a binary antipodalsignal in our implementation. The embedder generates CW by performing apointwise multiplication of W and CW. Other functions may be used togenerate CW from W and S, and the resulting signal need not be a binaryantipodal signal.

As shown in FIG. 1, the basic pattern, W, can also serve as anadditional security layer. In particular, the basic pattern may be usedto carry information about the media or an entity associated with themedia, such as its owner, the bearer of a security document, etc. In thespecific case of a security document, the system includes an OCR readerto capture text information carried on the document about the bearer,such as name, birthdate, address, ID number, etc. In the embedder ofFIG. 1, this data about the media object (101) is input to a hashfunction (103), which generates a hash. For example, in our securedocument example, this text information is input to a hash function,such as a CRC or secure hash, like MD5, SHA, etc This hash then formspart of a digital watermark payload message (104).

The embedder converts the payload into the basic pattern (106). Theprocess of generating a basic pattern from the payload can beimplemented in a variety of ways, and depends in part on the messagecoding process compatible with the digital watermark embedder. Forexample, some digital watermark embedders operate on binary signals,while others operate on M-ary symbols. One approach is to applyrepetition and error correction coding to generate an intermediatesignal from the payload, then spread the intermediate signal over abinary antipodal carrier signal using binary or M-ary spread spectrummodulation. The result is a binary antipodal signal that carries thepayload and is mapped to locations within the host media object.

The basic pattern may be integrated with a calibration signal or used inconjunction with a separate calibration watermark to compensate forgeometric/temporal distortion such as geometric/temporal scaling, shear,rotation, shifting, cropping, etc. For example, the carrier, in oneimplementation, is formed into a pattern that has a certain set oftransform domain peaks that enable geometric synchronization byperforming pattern matching between the peaks and a reference signal.

In one implementation, the embedder separately embeds the basic patternand the content dependent watermark using separate digital watermarkembedding operations 109, 110. One example for a secure document iswhere the basic pattern is embedded by modifying host image pixels at afirst resolution up or down according to the sign of the correspondingbinary antipodal signal element. The content dependent pattern is thenembedded similarly, but at a different spatial resolution. Both thebasic pattern and the content dependent pattern are embedded throughoutthe image and overlap. In an alternative example, the basic and contentdependent patterns are embedded at the same spatial resolution, but atmutually exclusive spatial locations (e.g., in interleaved pixelblocks). In general, the two watermarks are layered so as to minimizetheir interference; this can be achieved by embedding in discretespatial or transform domain features, locations, etc. As opposed to asimple binary quantization of a host signal value up or down, the hostsignal values or features corresponding to the watermark elements may bequantized to pre-determined bins or levels that adapt to host signalcharacteristics corresponding to the watermark element value. Also, thewatermark embedders may employ additional perceptual modeling to controlthe amount of variation to the host signal based on data hidingattributes of the host signal as measured using Human PerceptualModeling.

In another implementation, the embedder embeds only the contentdependent watermark (110), and it serves the dual function of bindingthe watermark to the host signal through its content dependencyattribute, and carrying other authentication information, such as thehash and a database pointer to a database entry storing informationabout the media object or the bearer of that object. One example of thisapproach is to invert the basic pattern only in selected locationscorresponding to the signature (e.g., where the signature has a valueof−1).

In yet anther implementation, the embedder embeds only the basic pattern(109), but does so using a content dependent quantization-based digitalwatermarking function, where the values of host signal elements arequantized into one of two sets of quantization bins, one correspondingto symbol 1 and another to symbol−1 of the binary antipodal signal.Alternatively, vector quantization may be employed in cases where thebasis pattern is coded in the form of M-ary symbols. Each possible M-arysymbol corresponds to a corresponding set of quantization bins. To embedthe basic pattern, the host signal values corresponding to elements inthe basic pattern are quantized into the closest bin of the setcorresponding to the symbol at that location in the basic pattern.

Returning generally to the process of FIG. 1, the embedder creates adigitally watermarked signal. In typical applications, this watermarkedsignal is rendered (e.g., printed or otherwise converted to analog form)(112). In our example of the security document, the security document isprinted and distributed to the bearer. As noted above, the media objectthen travels through a distortion channel (114), which occurs due to itsuse in the intended application.

FIG. 2 is a diagram illustrating a method for authenticating mediaobjects created using the method shown in FIG. 1. At various points inthe use of the media object, there are many instances where applicationsdemand automated verification of the object's authenticity, includingwhether the object itself is authentic, whether its bearer or owner iscorrect, etc. The layered security features implemented with the digitalwatermark enable such verification. In the case of secure documents,this authentication may be for access control to a place, facility,database, financial transaction, device, network system, etc. Theverification process may be overt, such as where a bearer of a documentis required to submit the document to a digital image scanner forverification. The verification process may also occur covertly, such aswhen a digital object passes through a node or gateway in a network, andis authenticated. Consider a case where the bearer of a credit cardpresents his credit card to a web camera to facilitate a financialtransaction on the Internet. An image captured on the card can beprocessed at a security gateway server, where the digital image of thecredit card is transmitted for digital watermark decoding and featureanalysis.

As shown in FIG. 2, the process begins with a digital version of themedia object 200, which is captured from its analog form or received indigital form. The specific operation varies depending on theimplementation of the embedder system.

As a first example, consider the case in which both the basic pattern,W, and the content dependent watermark, CW, are embedded. In the exampleof secure document captured by a digital camera or scanner, there islikely to be geometric distortion and cropping. As such, the detectoruses the calibration signal to synchronize with the basic pattern W. Thedetector then reads estimates of the basic pattern elements, W′, e.g.,using a reader compatible with the digital watermark embedder (202). Inour implementation, the reader applies a non-linear filter compatiblewith the embedder to characteristics of the media signal to estimate thevalues of the embedded pattern, W. It then performs de-modulation anderror correction decoding to recover the payload, including the embeddedhash, H. An error detection message in the payload may also be used toverify that that the payload has been recovered, error-free.

After getting the payload, the reader reconstructs the pattern, W, usingthe same technique as in the embedder.

In another processing thread or function, the verification systemcalculates the media signature, S′, (204) in the same manner as in theembedder. One of the benefits of using the calibration signal is that itenables the input signal to be calibrated (e.g.,geometrically/temporally aligned) before the signature is calculated.This aspect of the system provides greater flexibility and reliabilityto the signature calculation.

Next, the system computes CW as a function of W (or W′) and S′(208). Thenotation {CW′, W′ and S′} refers to the fact that these vectors may notbe identical to their counterparts in the embedder. A compatible digitalwatermark reader then extracts estimates of CW (210) from the mediaobject, which is preferably calibrated before extraction of CW. Thedegree to which CW can be extracted provides a first metric ofauthenticity. This measurement can be made by computing a correlationmeasure, and specifically, by a correlation measure between theextracted CW in block 210 and CW computed in block 208.

The measure of the content dependent pattern can be optimized bynormalizing or adapting it to media signal from which it is measured. Inone embodiment, the detector is programmed to normalize the measure ofcorrelation for CW by the strength of the extracted watermark, W′,detected in the media signal (e.g., the digital image scanned from aprinted object being authenticated). By normalizing the measure of CWrelative to the measurement of W′, the verification system achievesbetter differentiation of authentic and fake objects. Specifically, thestrength of W′ can be used to set a more effective threshold for themeasurement of CW in certain cases.

In the measurement of CW, there are two sources of error: 1. the errorbetween the original and re-computed signature in the received mediasignal; and 2 the error in extracting the watermark CW from the receivedmedia signal. In one implementation for printed images where theembedder inserts W and CW as primary and secondary watermarks atmutually exclusive locations in the host image and at the same spatialresolution in the host image, the measurement of the strength of theprimary watermark W provides a reliable predictor for the measurement ofthe secondary watermark. The detector uses the strength of the primarywatermark to set thresholds for the measurements of the secondarywatermark that specify which measurements of the secondary watermark aredeemed to be attributable to an authentic object and which areattributable to a fake. The rules for setting thresholds are preferablypredetermined based on empirical studies using statistical distributionsof signatures from authentic and fake host signals. Experiments showthat the separation between the distributions of the measurement of CWin originals and fakes gets stronger as the strength of the primarywatermark gets stronger. As these distributions separate from eachother, the thresholds indicating where fakes/authentic originals can bereliably distinguished widen as well. Based on tests on training sets,the implementer programmatically determines candidate thresholds for aparticular value of strength of the primary watermark. Then, duringoperation of the verification system, the detector adapts the thresholdfor CW based on the strength of W by selecting the appropriatethresholds as a function of W.

Further experiments show that differentiation between originals andfakes can be enhanced in cases where there is more bandwidth forembedding CW. In images, for example, the bandwidth for CW can beincreased for a fixed amount of perceptibility of the digital watermarkby increasing the amount of image data in which CW is embedded. Onespecific example is increasing the image area over which CW is embedded.This increase can be achieved by spreading and/or repeating the CWpattern over more image samples.

In addition, separation between originals and fakes can be increased byusing a longer signature. The effect of using a longer signature is thatit will be embedded less redundantly in the watermark that carries thecontent dependent pattern. Specifically, for a fixed number of samplesof the host media signal that are modified to embed CW, the redundancyof the signature decreases as the length of the signature increases.

The hash provides another layer of security. In our continuing exampleof a secure document, the personal information of the bearer on thesecure document, generally referred to as data about media 206, is inputto the same hash function used in the embedder 214, to create H′. Thispersonal data may include name, address, date of birth, height, weight,eye color, etc. This hash is then compared with the hash extracted fromW in block 216. The result is another indicator of authenticity (218),and in this example, indicates whether the personal information on thedocument has been altered. Even in the case where CW cannot beextracted, this measurement provides another indicator of authenticity.

The combination of the signature with the basic watermark provides anextra layer of security against photo ID card fraud, where one mightattempt to copy the watermark into his own photo and then place thatphoto along with a copy of the personal data from the authentic card ona fraudulent photo ID card. In this scenario, even if the hash in thewatermark matches the hash of the data on the card, the contentsignature will likely be different, and the measurement of the contentdependent watermark will indicate that the photo ID is a fake.

As noted above, there are alternative implementations of the system,corresponding to the alternatives described for the embedder above. Onealternative is where the basic pattern is stored or otherwise securelycommunicated to the verification system in a manner other than in thedigital watermark carried in the media object. This may be some othermachine-readable code in the secure document (e.g., 2D bar code,magnetic stripe, etc.), for example, or simply pre-programmed into theverification system.

Another implementation is where the signature, S, is used to transform(e.g., invert) selected portions of the basic pattern to create CW,without using a separate watermark to carry W. Note this transformationmay involve a simple inversion of the symbols, or a more sophisticatedscrambling or transform of the symbols in the base pattern correspondingto the signature elements. In this case, the verification systemcalculates S′, and then attempts to read W, with and without thetransform used to create CW. The result of these two read operations arethen compared, and should be drastically different if the media signalis valid, and closer if the media signal is invalid. The degree ofseparation that indicates that the media is not authentic is derivedthrough testing on training sets of valid and invalid objects. Theresult is a threshold test for the degree of separation between the twomeasurements.

Another alternative is to use an embedding and reading scheme for W thatis inherently content dependent. One such example is the quantizationscheme outlined above. In this type of scheme, the attributes of theembedding scheme make it difficult to extract W from one authenticdocument or object and insert it in another document or object withoutknowledge of the embedding methodology.

Concluding Remarks

Having described and illustrated the principles of the technology withreference to specific implementations, it will be recognized that thetechnology can be implemented in many other, different, forms. Toprovide a comprehensive disclosure without unduly lengthening thespecification, applicants incorporate by reference the patents andpatent applications referenced above inn their entireties, as if samewere fully set forth herein.

The methods, processes, and systems described above may be implementedin hardware, software or a combination of hardware and software. Forexample, the auxiliary data encoding processes may be implemented in aprogrammable computer or a special purpose digital circuit. Similarly,auxiliary data decoding may be implemented in software, firmware,hardware, or combinations of software, firmware and hardware. Themethods and processes described above may be implemented in programsexecuted from a system's memory (a computer readable medium, such as anelectronic, optical or magnetic storage device).

The particular combinations of elements and features in theabove-detailed embodiments are exemplary only; the interchanging andsubstitution of these teachings with other teachings in this and theincorporated-by-reference patents/applications are also contemplated.

1. A system comprising: one or more processors configured to: compute ahash of information on a media object; generate a pattern from the hash;compute a content signature from a media signal in the media object;combine the content signature and the pattern to form a contentdependent pattern; and embed the content dependent pattern as a digitalwatermark into the media object.
 2. The system of claim 1, wherein themedia object comprises a security document, and wherein the one or moreprocessors compute the hash from information on the security document.3. The system of claim 2, wherein the information on the securitydocument is printed on the document.
 4. The system of claim 1, whereinthe media object is a card, and wherein the one or more processors areconfigured to compute the hash from information comprising personal dataabout a bearer of the card.
 5. The system of claim 1, wherein the one ormore processors are configured to generate the pattern from the hash byspreading the hash over a carrier signal.
 6. The system of claim 5,wherein the carrier signal comprises a pseudorandom number.
 7. Thesystem of claim 1, wherein the one or more processors are configured togenerate the pattern from the hash based in part on error collectionencoding the hash.
 8. The system of claim 1, wherein the one or moreprocessors are configured to compute the content signature fromfrequency domain features of the media signal.
 9. The system of claim 1,wherein the media signal comprises an image, and wherein the one or moreprocessors are configured to compute the content signature from edgefeatures of the image.
 10. The system of claim 1, wherein the one ormore processors are configured to compute the content signature based inpart on filtering neighborhoods of samples within the media signal. 11.The system of claim 1, wherein the one or more processors are configuredembed the pattern and the content dependent pattern as separate digitalwatermarks into the media object.
 12. The system of claim 11, whereinthe media object comprises an image to be printed, and wherein the oneor more processors are configured to embed the separate digitalwatermarks at different spatial image resolutions in the image.
 13. Thesystem of claim 1, wherein the one or more processors are configured tocombine the content dependent pattern with the pattern by transformingelements of the pattern based on the content dependent pattern.
 14. Thesystem of claim 13, wherein the transforming elements comprises apointwise multiplication.
 15. The system of claim 13, wherein thetransforming elements comprises selectively inverting elements of thepattern based on values of the content dependent pattern.
 16. The systemof claim 13, wherein the one or more processors are configured to printthe media object.
 17. A system comprising: one or more processorsconfigured to: compute a first pattern; derive a content dependentsignature from a media signal in a media object; combine the contentdependent signature and the first pattern to form a content dependentpattern; and measure the content dependent pattern embedded as a digitalwatermark in the media signal to provide a measurement of authenticityof the media signal.
 18. The system of claim 17, wherein the one or moreprocessors are configured to compute the first pattern by extracting afirst digital watermark from the media signal.
 19. The system of claim18, wherein the first digital watermark carries a message payloadcomprising a hash of data about the media object, and wherein the hashis used in a second measurement of authenticity of the media signal. 20.The system of claim 19, wherein the media object comprises a printedobject and wherein the hash of data about the media object comprises ahash of data on the printed object.
 21. The system of claim 20, whereinthe one or more processors are configured to compute the hash from textdata printed on the printed object.
 22. The system of claim 18, whereinthe one or more processors are configured to extract the first digitalwatermark and the content dependent pattern as separate digitalwatermarks.
 23. The system of claim 18, wherein the first digitalwatermark comprises calibration attributes used to geometricallycalibrate the media signal before extracting the content dependentpattern.
 24. The system of claim 17, wherein the content dependentpattern selectively transforms elements of the first pattern.
 25. Thesystem of claim 24, wherein the one or more processors are configured toselectively transform elements of the first pattern by invertingelements of the first pattern.
 26. The system of claim 17, wherein theone or more processors are configured to generate the first pattern byextracting a digital watermark message from the media signal; performingerror correction decoding of the message, and combining the message witha pseudorandom number.
 27. The system of claim 17, wherein the one ormore processors are configured to: generate the first pattern from adigital watermark extracted from the media signal; measure a strength ofthe first digital watermark; and use the measurement of strength of thefirst digital watermark to adapt the measurement of authenticity basedon the content dependent pattern.
 28. The system of claim 17, whereinthe one or more processors are configured to print the media object. 29.A non-transitory computer-readable medium having instructions storedthereon, the instructions comprising: instructions to compute a hash ofinformation on a media object; instructions to generate a pattern fromthe hash; instructions to compute a content signature from a mediasignal in the media object; instructions to combine the contentsignature and the pattern to form a content dependent pattern; andinstructions to embed the content dependent pattern as a digitalwatermark into the media object.
 30. A non-transitory computer-readablemedium having instructions stored thereon, the instructions comprising:instructions to compute a first pattern; instructions to derive acontent dependent signature from a media signal in a media object;instructions to combine the content dependent signature and the firstpattern to form a content dependent pattern; and instructions to measurethe content dependent pattern embedded as a digital watermark in themedia signal to provide a measurement of authenticity of the mediasignal.